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' Abstract 

A method for computing lower bounds on capacities of 2-dimensional 
constraints having a symmetric presentation in either the horizontal or 
the vertical direction is presented. The method is a generalization of the 
method of Calkin and Wilf {SI AM J. Discrete Math., 1998). Previous 
best lower bounds on capacities of certain constraints are improved using 
the method. It is also shown how this method, as well as their method for 
computing upper bounds on the capacity, can be applied to constraints 
(/3 , which are not of finite-type. Additionally, capacities of 2 families of multi- 

O ^ ■ dimensional constraints are given exactly. 

Index-terms: Channel capacity, constrained-coding, min-max principle. 
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1 Introduction. 
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Fix an alphabet S and let ^ be a directed graph whose edges are labeled with 

■ symbols in S. Each path in Q corresponds to a finite word obtained by reading 
' the labels of the edges of the path in sequence. The path is said to generate the 

corresponding word, and the set of words generated by all finite paths in the 
graph is called a 1- dimensional constrained system or a 1-dimensional constraint. 
Such a graph is called a presentation of the constraint. We say that a word 
satisfies the constraint if it belongs to the constrained system. One-dimensional 

■ constraints have found widespread applications in digital storage systems, where 
\ they are used to model the set of sequences that can be written reliably to a 

medium. A central example is the binary runlength- limited constraint, denoted 
RLL((i, k) for nonnegative integers 0<d<k, consisting of all binary sequences in 
which the number of 'O's between conecutive 'I's is at least d, and each runlength 
of 'O's has length at most k. Another 1-dimensional constraint, often used in 
practice, is the bounded-charge constraint, denoted CHG(6), for some positive 
integer b; it consists of all words wiW2. . -Wi, where £—0, 1,2,... and each Wi is 
either -1-1 or —1, such that for all l<i<j<£, \ J2k=i'^k\^b. Other examples of 
1-dimensional constraints are the EVEN and ODD constraints, which contain 
all finite binary sequences in which the number of 'O's between consecutive 'I's 
is even and odd, respectively. Presentations for these constraints are given in 
Figure [TJ 
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Figure 1: Presentations of 1-dimensional constraints (a) EVEN |(b)| ODD; (c) 
CHG(5)p)]RLL(d, k). 



A 1-dimensional constraint over an alphabet S is said to have memory m, 
for some positive integer to, if for every word w of more than m letters over E, in 
which every subword of to+I consecutive letters satisfies S, it holds that w satis- 
fies S as well, and m is the smallest integer for which this holds. A 1-dimensional 
constraint with finite-memory is called a finite-type constraint. Among our ex- 
amples, RLL((i, k) is a finite-type constraint with memory fc, whereas EVEN, 
ODD and CHG(6) for b>2 are not finite-type constraints. 

In this work, we consider multidimensional constraints of dimension D for 
some positive integer D. These are sets of finite-size ©-dimensional arrays with 
entries over some finite alphabet specified by S) edge-labeled directed graphs. 
In Section [5] we give a precise definition of what we mean by J)-dimensional 
constraints. Here we just mention that they are closed under taking subarrays, 
meaning that if an array belongs to the constraint then any of its ©-dimensional 
subarrays consisting of "adjacent" entries also belongs to the constraint. As be- 
fore we say that an array satisfies the constraint if it belongs to it. Examples 
of multidimensional constraints can be obtained by generalizing 1-dimensional 
constraints, defining the constraint to consist of all arrays satisfying a given 
1-dimensional constraint S on every "row" in every direction along an "axis" of 
the array. We denote such a ©-dimensional constraint by S"*® . We will almost 
exclusively be concerned with 2 dimensional constraints, namely © = 2. In this 
case S"®^ is the set of all 2-dimensional arrays where each row and each column 
satisfy S. A well-known 2-dimensional constraint studied in statistical mechan- 
ics is the so called "hard-square" constraint. It consists of all finite-size binary 
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arrays which do not contain 2 adjacent 'I's either horizontally or vertically. 
Two variations of this constraint are the isolated 'I's or "non-attacking- kings" 
constraint, denoted NAK, and the "read- write-isolated- memory" constraint, de- 
noted RWIM. The former consists of all finite-size binary arrays in which there 
are no two adjacent 'I's either horizontally, vertically, or diagonally, and the 
latter consists of all finite-size binary arrays in which there are no two adjacent 
'I's either horizontally, or diagonally. Like their 1-dimensional counterparts, 
2-dimensional constraints play a role in storage systems, where with recent de- 
velopments, information is written in a true 2-dimensional fashion rather than 
using essentially 1-dimensional tracks. The RWIM constraint is used to model 
sequences of states of a binary linear memory in which no two' adjacent en- 
tries may contain a '1', and in every update, no two adjacent entries are both 
changed. See [2] and [3] for more details. 

Let S now be a ©-dimensional constraint over an alphabet S. For a S-tuple 
m = (mi,m2, . . - ^m^) of positive integers, Sm or Smix...xms denotes the set of 
all mi X 7712 X. . .xm^i arrays in S, and (m) denotes the product of the entries of 
m. We say that a sequence mi — {m^^^ . . . , m^-*) diverges to infinity, denoted 
mi oo, if (mj*'')^! does for each j. The capacity of S is defined by 

CMS) = lim i^f^, (1) 

where mi^cx), | • | denotes cardinality, and log — log2. The above limit exists 
and is independent of the choice of (mi)^-^, for any set S of finite-size J)- 
dimensional arrays over S which is closed under taking subarrays. This follows 
from subadditivity arguments (see 1 11] for a proof for X) = 2, which can be 
generalized to higher dimensions). In fact 

cap(5j = mt ; r — . (2) 

m (m) 

While a closed form formula for the capacity of 1-dimensional constraints is 
known (up to finiding the largest root of a polynomial), no such formula is 
known for constraints in higher dimensions, and currently there are only a few 
multidimensional constraints for which the capacity is known exactly and is 
nonzero (a highly non-trivial example can be found in . lOj ) . 

Let 5* be a 2-dimensional constraint over E, and m be a positive integer. 
The horizontal (resp. vertical) strip of height (resp. width) m of S, denoted 
HmiS) (resp. Vm{S)) is the subset of S given by 

Ti-miS) = y S„ixn (resp. V,„(5) = [J Snxm)- 
n n 

We show in Section[2]that 7im{S) and Vm{S) are 1-dimensional constraints over 
S". 

A method for computing very good lower and upper bounds on the capacity 
of the hard-square constraint is given in [T] (see also [TS]). Their method can 
be shown to work on 2-dimensional constraints for which every horizontal or 
every vertical strip has memory 1 and is "symmetric" ; that is, it is closed under 
reversing the order of symbols in words. The main contributions of our work 
are: 



3 



1. We establish a generalization of the method of [1 that gives improved 
lower bounds on capacities of 2-dimcnsional constraints, for instance for 
NAK and RWIM. 

2. We show how this generalization as well as the original method for ob- 
taining upper-bounds may be applied to a larger class of 2-dimensional 
constraints that includes constraints in which the vertical and horizon- 
tal strips are not necessarily finite-type. We illustrate this by computing 
lower and upper bounds on the capacities of the CHG(3)®^ and EVEN®^ 
constraints. 

3. We show that cap(CHG(2)®®) = 2"® and cap(ODD®®) = 1/2, for aU 
positive integers D. 

Previous work involving applications of the method of 1 and generalizations 
include [S], 0, [H], and [^l- 

2 Framework. 

In this section, we define the framework that we use in the rest of the paper. We 
deal with a directed graph G — (V,E), sometimes simply called a graph, with 
vertices V and edges E. For eGE we denote by ag{e) and Tg{e) the initial and 
terminal vertices of e in G, respectively. We shall omit the subscript G from 
ac and tq when the graph is clear from the context. A path of length £ in G 
is a sequence of £ edges {ei)f^iCE, where for i = 1,2, . . .,£—1, r(ei)=cr(ei+i). 
The path starts at the vertex cr(ei) and ends at the vertex T{ei). A cycle in G 
is a path that starts and ends at the same vertex Fix a finite alphabet S. A 
directed labeled graph Q with labels in S is a pair Q — (G, C), where G = (V, E) 
is a directed graph, and C : E \s a. labeling of the edges of G with symbols 
of S. The paths and cycles of Q are inherited from G and we will sometime use 
ug and Tg to denote tq and tq respectively. 

For a labeled graph Q = ((V, £'),£) with £ : i? — > S and a path {ei)l^i of 
Q, we say the path generates the word £(ei)£(e2). . ■C{ee) in E*. The graph Q 
is called lossless if for any two vertices u and v oi Q, all paths starting at u and 
terminating at v generate distinct words. The graph Q is called deterministic if 
there are no two distinct edges with the same initial vertex and the same label. 
Every 1-dimensional constraint S has a deterministic, and therefore lossless, 
presentation |14j . 

We introduce two 1-dimensional constraints defined by general directed 
graphs. Let G—(V, E) be a directed graph, the edge constraint defined by G, 
denoted X(G), is the 1-dimensional constraint over the alphabet E, presented 
hy Q — (G, Ie) where Ie is the identity map on E. For a graph G={V, E) with 
no parallel edges, the vertex-constraint defined by G, denoted X(G), is the set 

{{vi)UiCV:£^0,l,2,. . ., and for l<i<£, 3e£E s.t. a{e)^v^, T{e)=v,+i} . 

It's not hard to verify that vertex-constraints and edge-constraints are 1- 
dimensional constraints with memory 1. In fact, the vertex constraints are 
precisely the finite- type constraints with memory 1. And it can be shown that 
edge constraints are characterized as follows. The follower set of a symbol a in 
a constraint S is defined to be {6 : a& G 5}; edge constraints are precisely the 
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constraints with memory 1 such that any two follower sets are either disjoint or 
identical [T^ exercise 2.3.4]. 

We consider multidimensional arrays of dimension £)-a positive integer. 
For an integer I denote by [P\ the set {0,1,...,^—!}. For a S-tuple m = 
(mi, . . . , mj)) of nonnegative integers and a finite set A, we call an mi x m2 x 
... X mg !D-dimensional array with entries in j4, a 2)-dimensional array of size 
m over A. We shall index the entries of such an array by [mi] x \m2\ x . . . x [mj)] . 
Let A™ or ^mix -x™© denote the set of all S-dimensional arrays of size m over 
A. We define A*---* = A* , where the number of '*'s in the superscript is D, by 

A*"" = (JS"", 

m 

as the set of all finite-size D-dimensional arrays with entries in A. Let T&A* be 
such an array. Given an integer l<i<!D, a row in direction i of F is a sequence 
of entries of F of the form [ki,k2.....,ki-i:3M+i,--Mv))'^lo ^o^' so™*^ integers 
ki(z.[mi\\ 1<Z<2), l^i- In this paper, for J) = 2, we use the convention that 
direction 1 is the vertical direction and direction 2 is the horizontal; thus the 
columns of a 2-dimensional array are its rows in direction 1, and its "traditional 
rows" are its rows in direction 2. Let i? be a finite set and C : A ^ B he 
a mapping. We extend C to a mapping C : A* -^B* as follows. For a ID- 
dimensional array FgA™, £(F) is the array in obtained by applying £ to 
each entry of F, that is 

(£(F))j = £((F)j) , jG[mi]x...x[ms]. 

Additionally, for a subset ^C^*^ we define C{S) = {C{T) : TeS}. 

We generalize the definition of constrained system to D dimensions. Let 
g = (^1,^2, ■ ■ Ss), be a D-tuple of labeled graphs with the same set of edges 
E and the same labeling C : E ^ T^. The edge e has D pairs of initial and 
terminal vertices {crgi{e), rg. (e))-one for each graph Qi in Q. We say that an 
array FeS* of size m is generated by G if there exists an array T'gE*^ of 
size m, such that for i = 1, 2, . . ., J), every row in direction i of F' is a path in 
Gi, and C{T') ^T. We caU the set of all arrays FeS*® generated by G, the 
^-dimensional constrained system or the 'S -dimensional constraint presented 
by G, and denote it by X(5). We say that ^ is a presentation for X(5). 

In [7], 2-dimensional constrained systems are defined by vertex-labeled 
graphs, with a common set of vertices and a common labelling on the ver- 
tices. It can be shown that their definition (generalized to higher dimensions) is 
equivalent to ours. We find it more convenient to use our definition, since, just 
as in one dimension, it permits use of parallel edges and often enables a smaller 
presentation of a given constraint. 

Figured] shows presentations for the NAK and RWIM constraints defined in 
Section [TJ In these presentations Gi and G2 describe the vertical and horizontal 
constraints on the edges, respectively. Each edge e = {e)ij is a 2x2 binary 
matrix of the form 

( (e)(o,o) (e)(o,i) \ 
V (e)(i,o) (e)(i,i) / ' 

and it is labeled by (e)(i 1), i.e., the labelling of an edge simply picks out the 
entry in the lower-right corner. For NAK, the edges E = -Enak are the 2x2 
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matrices with at most one 1; note that a rectangular array of any size satisfies 
NAK iff each of its 2 x 2 sub-arrays belongs to E; similarly, for RWIM, the edges 
E — i?RwiM are the elements of i?NAK together with 



1 
1 



and 



1 
1 



and again a rectangular array of any size satisfies RWIM iff each of its 2 x 2 
sub-arrays belongs to E. In the figures, each edge is drawn twice-once in Qi 
and once in 52~and the matrix identifying it is written next to it. The states 
are 1x2 blocks for Qi and 2x1 blocks for Q2', for an edge e, 



and 



c^ei(e)= (e)(o,o)(e)(o4) > andrg;,(e)= (e)(i,o) (e)(i,i) , 
ag^ (e) 



(e)(04) 



, andr,,(e) 

It follows that, for both constraints, and any rectangluar array r'si?™'*" with 
each of its rows a path in Q2 and each of its columns a path in Qi, it holds that 




for i — l,2,...,m— 1, j — l,2,...,n— 1. Therefore, the only 2x2 sub-arrays 
appearing in the array C{T') are elements of E, and it follows that £(r') sat- 
isfies the corresponding constraint. Similarly, it can be shown that any array 
satisfying the constraint can be generated by the presentation. 




(a) 




Figure 2: Presentations of 2-dimensional constraints: (a) NAK constraint; |(b)| 
RWIM constraint. 



The axial product of D sets Li, . . .,Lx)CE*, denoted Li®L2'8'- • -^-^s ^ S* , 
is the set of all arrays FeS* such that for i = 1,2,...,D every row of 
r in direction i belongs to Li. If Li=L2=. ■ ■=L^ = L we say that the 
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axial-product is isotropic and denote it by L®®. Given a presentation Q — 
((Gi,£), . . ., (Gj),/!)) for a S)-dimensional constraint S with a common set of 
edges E, the set X(Gi)(8)- ■ .i8)X(Gs)) C E'* is a S-dimensional constraint pre- 
sented by ((Gij/fi), . . ., {G^,Ie)), where Ie is the identity map on E. 

We say that G is capacity-preserving if cap(X(Gi)(g). . .(^X{G^)) = cap(S'). 
For S) = 1 any lossless, and therefore deterministic, presentation of S is capacity- 
preserving, since in a lossless graph with \V\ vertices there are at most \V\'^ paths 
generating any given word; for !D > 1 , the question whether every ©-dimensional 
constraint has a capacity-preserving presentation is open. This is a major open 
problem in symbolic dynamics, although it is usually formulated in a slightly 
different manner; see where it is shown that for every ©-dimensional con- 
straint S and e > 0, there is a presentation Q = ((Gi, £),..., (G®, £)) such 
that cap(S')<cap(X(Gi)(8). . .(g)X(Gx)))<cap(S')-|-e. We show in the next propo- 
sition that the answer to this question is positive, if S is an axial product of © 
one-dimensional constraints. 

Proposition 1. Let Si, 82, ■ ■ ■, S^Q'E* be D one- dimensional constraints over 
E, and let S = Si®...®S^, then S is a ^-dimensional constraint over S. 
Moreover, S has a capacity-preserving presentation. 

Remark . There arc ©-dimensional constraints which are not axial products of © 
one-dimensional constraints. For example for ©=2, the NAK constraint defined 
in Section [1] is not an axial product. 

Proof. Let Gs, = ((Fi, Si), £1), ^s, = {{¥2, E^), C2), . . .^Gs^ = {{V^,E^),Cs) 
be presentations of 5*1, . . ., S"©, respectively. Define the ©-tuple of labeled graphs 
Q = {Gi, ■ . ■,G's), as follows. Let 

E= l^iei, . . .,es) €Y[E, : Ci{ei)=C2{e2)^. . .=C^{es)^ , 

and let £ : -E S be given by 

£(ei, . . .,ex,) = £i(ei) ; {ei, . . .,es)&E. 

For i = 1,2, . . .,©, the graph G^ is defined by Gi = (Gi,£) with d = {¥,E), 
where for e — (ei, . . .,es)(zE, (Tg.{e) = ag^,. (e^) and Tg.{e) ~ rg^. (e^). It's easy 
to verify that is a presentation of Si(E)S2'E). ■ .^S^. 

Assume now that every Gg. is losseless. We show that in this case G is 
capacity preserving. Let X=X(Gi)(E). . .(S)X(Gx)), n be a positive integer, and 
let n be the ©-tuple with every entry equal to n. We extend the mapping £ to 
£ : Xn-^Sn as described above. Now, fix an array FeS'n, and for i = 1, 2, . . ., © 
let r'(*)G(i?j)" be an array such that every row in direction i, (rj|^*^)^^j is a 
path in Gst generating the corresponding row (Fj,^)^^^ in F. Let F'eE" be the 
array with entries given by Fj — (Fj''^\ . . .,Fj^®''), It follows from the 

construction of G that F'eXn, and that £(F') — F. Moreover, any array AeX„ 
such that £(A) = F can be constructed in this manner. Now, as each Gsi is 
lossless, there are at most \ Vi\'^ possibilities of choosing each row in direction i of 
F'('\ and as there are n^~^ such rows, there are at most ^ possibilities 

of choosing each F'^*) . It follows that 

4=1 
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where C-^{{T}) = {r'eX„ : jC{T') = T}. Summing the latter inequahty over 
all re«Sn, we obtain 

\Xn\<\Sn\l[\Vif"''~\ 

i=\ 

Taking the log, dividing through by n®, and taking the limit as n approaches 
infinity, we have cap(X)<cap(S'). Clearly, cap(X)>cap(6'), since X is a presen- 
tation of S. The result follows. ■ 

A graph G = {V, E) is irreducible if for any pair of vertices u,v€V there is 
a path in G starting at u and terminating at v; it is primitive if it is irreducible 
and the gcd of the lengths of all cycles of G is 1. We denote by A{G) the 
adjacency matrix of G: namely the \V\ x \V\ matrix where (A{G))^j is the 
number of edges in G from i to j. We use 1 to denote the real vector each 
of whose entries is 1. For a nonnegative matrix A denote by X{A) its Perron 
eigenvalue, that is, its largest real eigenvalue. It is well-known that for a 1- 
dimensional constraint S presented by a lossless labeled graph Q = (G, £), 
the capacity of S is logA(A(G)). In particular, for a graph G, it holds that 
cap(X(G)) = logA(A(G)). We say that a graph G is symmetric if A{G) is 
symmetric. Wc say that a vertex of a graph is isolated if it has no outgoing 
nor incoming edges. We say that a vertex-constraint (resp. edge-constraint) is 
symmetric if it is defined by a symmetric graph. For a vertex-constraint, this 
definition is equivalent to requiring that the constraint is closed under reversal 
of the order of symbols in words. Note, that in a symmetric edge-constraint, up 
to removal of isolated vertices, the (unlabeled) graph defining the constraint is 
unique. 

In this paper, we will mostly deal with 2-dimensional constraints. For a 

2-dimensional array reS™^^™^, for nonnegative integers mi, m2, we denote 
by r'^ its transpose, namely (r'^)(jj) = (*,j)s["^i]x['Ti2]. For a 

2-dimensional constraint S over E, we use 5* to denote the set 

= {res** : t^gS} , 

Clealry 5* is a 2-dimensional constraint with cap(5'^) = cap(5). 

Let 5 be a 2-dimensional constraint over an alphabet A, and consider a 
horizontal (resp. vertical) strip Hm{S) (resp. Vm{S)) of S for some positive 
integer m. We regard such a strip as a set of 1-dimensional words over 
where each mxn (resp. nxm) air ay in the strip is considered a word of length 
n over A^. Below we show that the horizontal and vertical strips of S are 
1-dimensional constraints over A^. For this, we need the following definition. 
Let G = (y, E) be a graph, and let m be a positive integer. Let G^™ be 
the graph given by G^™ = {V''",E"'), where for each e = (ei, . . e„i)ei;™, 
crGx™(e) = ((7G(ei), . . .,(TG{em)) and TGx™(e) = (rG(ei), . . ., rG(em)). For a 
labeled graph g = {G,C) with G = {V,E) and C : E ^ T,, let be the 
labeled graph defined by Q''"' = (G^™, r^™), where : i;™ ^ is given 
by 

£'<"'{ei,...,e,n) = mei),...,£{em)) ; (e^, . . ., e,n)eE"^ 

We call G^™ (resp., CJ^™) the mth tensor-power of G. (resp., G). We can now 
state the following proposition. 
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Proposition 2. Let S be a 2-dimensional constraint over E and let m be a 
positive integer. Then 

1. Ti.m{S) (resp. VmiS)) is a I -dimensional constraint over E™. 

2. Let S = T(^)(g)TC^) for I -dimensional constraints T^^\ T^^^ over E, pre- 
sented by labeled graphs Q'^^\ Q'^'^\ respectively. Then the 1- dimensional 
constraint Tim (S) is presented by the labeled graph Qm defined as the sub- 
graph of the labeled graph consisting of only those edges whose 
label (an m-letter word over T,) satisfies T^^\ An analogous statement 
holds for VmiS), with respect to the graph Qm^ formed in a similar way 
from (^;W)X'". 

Proof. It suffices to prove tliis only for liorizontal strips HmiS). We first prove 
part m It's easy to verify that tlie labeled graph presents the con- 

straint over E™, consisting of all mxn arrays of E** such that every row satisfies 
T^^\ It follows that the sub-graph Qm\ formed by removing all the edges of 
(^g{'H)^xm are labeled with a word that does not satisfy T'^\ presents the 
1-dimensional constraint consisting of all mxn arrays with every row satisfying 
r(^) and every column satisfying T'^'. This is precisely the constraint Hm{S). 

We now prove part [TJ Let the pair of labeled graphs (t/'-^-' , tJ^^-*) be a 
presentation of S, where G^^^ = {{V^^\E),C) and G^^'^ = {{V^^\E),C). De- 
fine the edge-constraints = X{V^^\E) and f^^) = X{V^'^\E). Since 
(g(V) is a presentation of S, we have S = C{£^^^(g>£^^'>), and therefore 
HmiS) = CiHmi£^^'^(S>£^'^^)). By part El Hmi£^^'>(8>£^'^'^) is a 1-dimensional 
constraint, presented by a labeled graph Gm with edges labeled by words in 
E™. Replacing each such label eGE'^ in that graph with £(e) we clearly obtain 
a presentation of = HmiS). ■ 

We shall use the following notation for 2-dimensional arrays. Let A be a set 
and TgA"^^. For integers 0<si<S2<s and 0<ti<t2<t, we denote by Ts-^.s2,ti:t2 
the sub-array: 

(rsi:S2,ti:t2)ij = rsi+j,ti+j ; iiJ)e[s2-Sl + l]x[t2~tl + l], 

andhy Tsi:s2^* (resp. r*,t^:tj the sub-array rsi:s2,0:t-i (resp. ro:s-i,ti:t2)- We 
also abbreviate x:x in the subscript by x. We shall use the same notation for 
one-dimensional vectors: for a vector veA*, v^-.t denotes the subvector 

(vs:t)j = Vs+i ; ie[t-s-\-l]. 

3 Constraints with symmetric edge-constrained 
strips. 

In this section we generalize the method presented in fT\ to provide improved 
lower bounds on capacities of 2-dimensional constraints whose horizontal strips 
are symmetric edge-constraints. 

Fix an alphabet E, and let 5 be a 2-dimensional constraint over E. We say 
that S has horizontal edge-constrained-strips if for every positive integer m, the 
constraint TimiS) is an edge-constraint. If, in addition, every horizontal strip 
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is symmetric, we say that S has symmetric horizontal edge- constrained strips. 
Analogously, using Vm(S'), we have the notions of a 2-dimensional constraint 
with vertical edge-constrained-strips and symmetric vertical edge-constrained- 
strips 

Here, we consider constraints of the form S = T ® £, where E—XiGg) 
is an edge-constraint defined by the graph — {Vs^Eg) and T is an arbi- 
trary 1-dimensional constraint over S. Then £ is presented by Qg — {G£,Ie) 
where Ie is the identity map on Eg. Let m be a positive integer. By Propo- 
sition O part O TLmiS) is a 1-dimensional constraint presented by a subgraph 
g^^^ = (Gif\/|") of e^^™. It follows that n^{S) = X(G^^), and so S has 
horizontal edge-constrained strips. Henceforth, we further assume that it has 
symmetric horizontal edge-constrained strips; note that symmetry of the graph 
Gg is necessary but not sufficient for this assumption (see Proposition [3] below). 

For a positive integer m, let F„i = KVg)™!, and let H„i denote the F„i x Fm 
adjacency matrix of Gm . Since the limit in ([T]) exists independently of the 
choice of (mi)^^, and since lim„^oo log(|5mxn|)/n- = cap(7Ym(5)) for every 
positive integer m, we have 

cap(Dj = lim j mn 

ni,n — >oo 

l0g|S'mx«| 

= lim lim 



— lim 



oo ri— ►oo rnn 



m 



= lin, i^lW^O. (3) 

m — ^oo fii 

For a matrix M, let M"^ denote its transpose. Fix a positive integer m. Following 
[T], since Hm is real and symmetric, we obtain by the min-max principle 



ymym 



for any Fm x 1 real vector y„i =/= and positive integer p. Choosing ym to be 
the vector iJ^'^x^, for some positive integer q and F„i x 1 real vector x„i such 
that Ym 7^ 0, we have 

t TT2q+p^ 

mrn) > ^"4772^- (4) 



Thus by Q, it follows that 



1 1 X* H^'J+P-x 

cap(5) > - lim sup - log -. (5) 

p m^oo m xt,iJ„?x™ 

In [T], each x™ is chosen to be the Fm x 1 vector with every entry equal 
to 1. We obtain improved lower bounds in many cases by choosing other se- 
quences of vectors, {xm)m=ij as follows. We fix integers ^>0 and a>l, and 
let (j) : (Vf:)''^" [0,oo) be a nonnegative function. Our method works for se- 
quences (xto^)^-^ = (z^j^)^_^ where ^ ji + ka for positive integers fc, and 
is the Fm^ xl nonnegative vector indexed by (Vg)™'': 

k~l 

«)v=n'^(^— ; ve(l/£)™'=. (6) 

i=Q 
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For such sequences and a fixed positive integer n, we will show that one can 
compute Ln, the growth rate of x*„^i7"j^Xmj. : 

L„ = hm ^"g"--^--"-, 
fc— *oo rrik 

and from ^ we obtain the lower bound cap{S)>{L2q+p — L2q)/p- 

Before doing this for general /i and a, it is instructive to look at the special 
case: ji — and a — 1. In this case ruk = k, and 

k-l 



Let {xrn)m=i = i'^m)m=i ^^'^ ^^t n be a positive integer. For a word 
w = wi. . .WnG£ define its weight, W^{w), by 'Wcj,{w)—(j){a(wi))(f){T{wn)), where 
wi , Wn are regarded as edg es in cind extend this to arrays ^^Smxn by 

?7l — 1 

W0(r) = n w^r.,:)- 

1=0 

Observe that for an array T€Smxn that is a path in G^'' of length n starting at 
ve(V£)" and ending at ue(y£)™, it holds that W^{T)={zt)^,{2t)u■ It follows 
that 

Now, pick a deterministic presentation, 5i^'' = (vi^'', -E'i^'', /zi^'') of V„(S'), and 
let : E^[0,oo) be the edge weighting defined by ^^(e) = W^{C^n\e)) for 
eei;4"^\ Let A(g^^\>V^) be the |yi^^|x|yi^^| weighted adjacency matrix of 
5^^^ with entries indexed by vS^^ x vj^^^ and given by 



Then 



where the sum is taken over all paths 7 in of length m and (7) denotes 
the array in Smxn generated by 7. Since is deterministic it follows that 

By equation ([7]) the RHS is L„ and by Perron-Frobenius theory the LHS is 
logA(A(5i^', W^)), and thus i„ = logA(A(d^', W^)). 

For general /i, a, we proceed similarly. We pick a deterministic presenta- 
tion = {V^\ C^n ^) of V„(S') and construct a labeled directed graph 
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J=J{fi, a, n, Qri^\G£)—{Vi, Ex, Cj), with nonnegative real weights on its edges 
given by : Ej — > [0,oo). The graph X and weight function are defined 
as foUows. The set of vertices Vj is given by 

and the function Cj : Ex j]"^" labels each edge with an axn array over 
S. We specify the edges of I by describing the outgoing edges of each of its 
vertices along with their weights. Let v = (f , z;, 1) S Vj be a vertex of I. The 
set of outgoing edges of v consists of exactly one edge for every path of length 
Oi in Qri starting at v. Let 7 = (ei)"^Q^ C E^'^ be such a path and let u be 
its terminating vertex. We regard the word generated by 7 in Q^'^ as an array 
F e with entries given by (F),,, = (c'^n\e^)) , Let f = (/o, . . . , /^-i) and 

1 = (/q, . . ■ , l^i-i) and for i — . . ., /i+a— 1, define fi to be (T(Fi_^_o) and U 

to be t(F,;_p,„_i), where Fi_^,o and Fi_p,„_i are regarded as edges in the graph 
Gg. For such a path 7 the corresponding outgoing edge e G i?i of v satisfies 

(j{e) = V, £1(6) = F, T(e) = ((/a,/a+l,...,/Q+p-l),W, (^a,Za+l,...,Za+^-l)). 

The weight of e, >V0(e), is given by W0(e) = (?!)(/o, . . . , /^+q_i)(?!)(/o, • • ■ 
We shall regard the label of a path (ei)fZQ in X as the laxn array F over E 
resulting from concatenating the labels of the edges of 7 in order in the vertical 
direction, namely Tia+k.j = {^iiei))k.j, for all iG[£], kG[a] and jG[n]. Finally, 
we define the weighted adjacency matrix of the labeled directed graph I with 
weights given by as the \Vi\ x \Vx\ nonnegative real matrix A{1,W^) with 
entries indexed by (Vj)^ and given by 

The following lemma generalizes ideas in [1] and uses the weighted labeled 
graph X to compute L„, when (x^J^^ = (z,^jj^i. 

Lemma 1. i^or (x„jj^^ (^tj^li' andJ=I{fi,a,n,g^\Gs), 
^■^ lQg<».gm.x.„, ^ logA(^(T,>V^)) 

Proof. We shall show that there are positive real constants c, d such that for all 
positive integers fc, 

c • 1* 1 < ^l^HZ,^„,, <d-l' {A{I,W^)f 1. (8) 

For a positive integer s and vector e = (eg, . . . , es-i) in {EgY denote by 
cr{e), r(e) G {VeY the vectors with entries given by 

(r(e))^. = r(e,) i ' ' ' J 

Now, fix a positive integer k. Let F be an array in Sm^xn- Recall that each 
entry of F is an edge in Gg and define the weight of F, denoted >V0(F), by 

fe-i 

W0(F) = Yl_'P{'^(^ia:ia+tJ.+a-l,o))<l>{T{Tia:ia+ti+a-l.n-l))- 
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For a positive integer I, let Vi denote the set of paths of length £ in 2. We 
denote the label of a path ^^Vi by It is easily verified that there exists 

a path in Vi with label F g j]^"^" if and only if there exists a path in C/i^-* of 
length £a that generates F. As Gr^^ is a presentation of Vn, the set of labels of 
paths in Vi is Siaxn- 

For a finite path 7 in T, define its weight, denoted 'W^{'-f), as the product of 
the weights of the edges in the path. Recalling that the entries of x™^ = z^^ 
are indexed by (Vg )'"'', we observe that 



For an array F G S^rt^ y.m we say that a path 7 € Tnatchcs F if it is labeled by 
the sub-array F^:mj,_i_, and starts at a vertex (f , v,i) G Vj with f = (t(Fo:^_i,o) 
and 1 = r(Fo:;i_i,„_i). It can be verified from the construction of X that if 7 
matches F then ^0(7) = W0(F). 

Now, since gIi^^ is a presentation of V„(S'), it follows from the construction 
of X that every F S Sm^xn has a path in Vk matching it. Conversely, since 
for a path 7 G Vk all arrays F G Smi^xn that it matches have the same sub- 
array F^:„ij,_i^*, it follows that there are at most jSj^" arrays in Sm^xn that 7 
matches. Therefore, 

This shows the right inequality of ([H) , we now turn to the left . Set k' = fc+ [/i/a] , 
s = \fi/a\a~- fi, and let ip -.Vk' ^ Sm^yn be given by 

For a path 7 e Vk', with 7 = (ejjLo^ C Ex, its weight satisfies 

fc'-l /\tJ./a]-l \ 

>V^(7) = [] W^e,) = n ^^(^-(7)) < $'rM/-l>v^(^(^))^ 

where we take $ to be a positive constant satisfying $> max {0(v):ve(V£)^+"}. 
Now let F be an array in Sm^xn- Since gIi^^ is deterministic, so is X, and thus 
for every vertex v G Vj, the paths in V'f. starting at v are labeled distinctly. As 
all paths 7 that map to F under ip have the same sub-array {Cx{'^)) ^, 
it follows that there are at most paths ^ € P^. starting at v such that 

'0(7) = F. Consequently, there are at most |Vi||S|'*" paths in Vk' that map to 
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r under ip. Therefore, 



< 



Ten- 

^2rp/ai ^ n;4V(7)) 

Dividing both sides by $2rp/al jl/^;! we obtain the left inequahty of ©. The 
claim of the lemma now follows from Perron- Frobenius theory by taking the log 
of ([8|), dividing it by nik and taking the limit as k approaches infinity. ■ 

We thus obtain the following lower bound on the capacity of a 2-dimensional 
constraint. 

Theorem 1. Let T,£ he 1- dimensional constraints over an alphabet E, with 
£ an edge constraint defined by a graph Gs = (VsjEs). Set S = T®£ and 
suppose that S has symmetric horizontal edge- constrained strips. Let /i>0 and 
a,p,q>0 be integers and (j> ■ (Vg)^^" —> [0, cxd) be a nonnegative real function. 
For a positive integer n let Qn be a deterministic presentation ofVn{S) and set 
An,4, ^ A{I{fi,a,n,g„,G£),W^). Then 

> logA(A.,+,,,)-logA(A,,,,)^ 
pa 

Remark 1. In addition to computing lower-bounds, [T] gives a method for com- 
puting upper bounds on the capacity of the hard-square constraint. It can be 
shown that this method can also be applied to all constraints of the form T^£, 
with £ an edge constraint, having symmetric horizontal edge-constrained strips. 

Remark 2. Theorem [1] can be generalized to apply to 2-dimensional constraints 
having symmetric horizontal edge-constrained strips, which are not necessarily 
axial-products. Let S be such a constraint, and for every positive integer m, let 
G^m'' — {vS!~^\ eIu''^) be the symmetric graph, with no isolated vertices, defining 
UmiS). Set Gs = G^^\ Vs = V^^'^ and Eg = E^^K We claim there exists 
a mapping : V^m^^— >(Vf such that for every edge e = eoCi. . .em-i'^Eln'\ 
with each Ci^Eg, 

fm{cr{e)) = (cr(eo), . . ., (T(e„_i)) and /„(T(e)) = (r(eo), . . ., r(e™_i)). (10) 

This mapping is defined as follows. For a vertex vGVrk , pick an incoming edge 
e = eoei. . .em~i£Em and define fm{v) as (T(eo), . . .,T{e„i-i))- This mapping 
is uniquely-defined: indeed if e' = Cq. . .e'^^_]^G-Ern is another incoming edge 

Ch/) 

of V, and g = go- ■ -gm-i&Em is an outgoing edge of v, then clearly, for every 
iG[m], both etgi and e[gi are paths in Gg ; consequently T{ei) = T{e[). It is easy 
to check that /,„ satisfies the conditions in PH)) . Now, replace the definition of 
zl, in ® with 

fc-i 

i=0 
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With this new definition and the aid of pU|) . it can be verified that Lemma [T] 
and consequently Theorem [T] still hold. 

Remark 3. Clearly, it is sufficient, for the theorem to hold, that TCmiS) is 
symmetric for large enough m. 

We now give a sufficient condition for the constraint S — T®8 to have sym- 
metric horizontal edge-constrained strips. For this to happen, we (generally) 
must have that Gs is symmetric. This means that there exists a "matching" 
between edges, were each edge is matched with an edge in the "reverse" direc- 
tion. More precisely there is a bijection R : Eg ^ Eg such that for all e ^ Eg, 
((T(e),T(e)) = {T{R{e)),a{R{e))) and R{R{e)) — e. We call such a bijection an 
edge-reversing matching, and we denote by TZ{Gg) the set of all edge-reversing 
matchings of Gg. Clearly a graph G is symmetric iff it has an edge-reversing 
matching. Thus T(g>£ has symmetric horizontal edge-constrained strips iff for 
every m, Qm has an edge-reversing matching. We present a sufficient condition 
for this to hold. 

Proposition 3. Let T,£ be 1- dimensional constraints over an alphabet E, with 
£ an edge constraint defined by a graph Gg =^ {Vg,Eg) with R G TZ{Gg) an 
edge-reversing matching. If for all positive integers m and words e\. . .em&Eg^ , 
if e\. . .Cm satisfies T then R{ei) . . .R(em) satisfies T as well, then T®£ has 
symmetric horizontal edge- constrained strips. 

Proof. Let G\n^ = {V^,E^n'^) be the sub-graph of G^™ that defines H„i(S'). 
We show that G^' is symmetric. Let i?^™ : E'^ be defined by 

i?^"(ei,...,e„) = (i?(ei ),..., i?(e„0). 

Clearly, i?^™ is an edge-reversing matching of Gg^^. Recall that eI^^ consists 
of all the edges in Eg^ that, when regarded as m-letter words over S, satisfy T. 
Therefore, by the assumption, it follows that for all eEEm , R^'^{e)EEm as 
well. Consequently, R^"^ restricted to Em , is an edge- reversing matching of 
Gin and hence it is symmetric. ■ 

If Qt is a presentation of T and RGTl{Gg), a sufficient condition for the hy- 
pothesis of Proposition[3]to hold, which may be easier to check, is the existance 
of a function / : Et — * Et satisfying: 1) Crifie)) — R{CT{e)) for all e € Et 
and 2) for any path 6162 of length 2 in Qt, the sequence /(ei)/(e2) is also a 
path in Qt- Indeed, if such a function exists then any path €162. . .Cm in Gt 
generating a word 6162. . .e™ has a corresponding path /(ei)/(e2)- ■ -/(em) gen- 
erating the word i?(ei)i?(e2). . .i?(em), and thus the hypothesis of Proposition [3] 
is fuUfiled. In fact, it can be shown that when Qt is irreducible, deterministic 
and has the minimum number of states among all deterministic presentations 
of T, this condition is also necessary for the hypothesis of Propositon[3]to hold 
(see [131 Section 3.3]). 

In Section [5] we use Proposition [3] to show that the method described in this 
section can be used to compute lower bounds on CIIG(fei)(8)CHG(62) for any 
positive integers 61 and 62- 
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4 Constraints with symmetric vertex-constra- 
ined strips. 

In this section we present an analog to Theorem [T] that gives lower bounds on 
the capacities of constraints for which every horizontal or every vertical strip is 
a symmetric vertex-constraint. We do this by transforming a 2-dimensional con- 
straint with symmetric vertex-constrained strips to a 2-dimensional constraint 
with symmetric edge-constrained strips, having the same capacity. 

Fix an alphabet E, and let S* be a 2-dimensional constraint over E. We say 
that S has horizontal vertex- constrained strips if for every positive integer m, the 
constraint Ti.m{S) is a vertex-constraint. If, in addition, every horizontal strip 
is symmetric, we say that S has symmetric horizontal vertex-constrained strips. 
The notions of a 2-dimensional constraint with vertical vertex-constrained strips 
and symmetric vertical vertex-constrained strips are defined analogously. 

It turns out that RWIM and NAK do not have horizontal or vertical edge- 
constrained strips, and so the method in section [3] docs not apply directly. We 
illustrate this only for horizontal strips for S = RWIM. Recall from section [2] 
that an edge constraint is a constraint of memory 1 such that any two follower 
sets are either disjoint or identical. We claim that this condition does not hold 
for TimiS). To see this, given any m, let w be the all-zeros word of length m 
and u ^ w he any other word of length to. Now, the to x 2 arrays 

Wo Wo Wo Uo "0 Wo 

Wi Wi Wi Ui Ml Wi 

7 5 ; 

Wm_l Wm-1 Wm-1 "m-l Um-1 Wm-1 

belong to S, yet the to x 2 array 

Uo Uo 

Ui Ui 



does not. Thus, w and u, viewed as to x 1 columns have different but non- 
disjoint follower sets. Thus, RWIM does not have horizontal edge-constrained 
strips. 

However, it is not hard to show that RWIM and NAK have both symmetric 
horizontal vertex-constrained strips and symmetric vertical vertex-constrained 
strips. For instance, for S = RWIM, HmiS) is the vertex constraint defined 
by the graph G — {V, E), where V consists of all binary vectors uq ■ • ■ "m-i of 
length TO and E consists of a single edge from u G V to v G ^ iff for all i, 
whenever Ui = 1, then w^+i — Vi = Vi-i = (with the obvious modification 
when i = or TO — 1). And Vm{S) is the vertex constraint defined by the graph 
of G" = {V, E'), where V consists of all binary vectors uo ■ ■ ■ Um-i of length 
TO which do not contain two adjacent 'I's and E' consists of a single edge from 
It e to w G iff for all i, whenever Ui — 1, then w^+i = Vi-i ~ (again, with 
the obvious modification when i = or to — 1). Clearly, both G and G" are 
symmetric. 

Now, let 5 be a 2-dimensional constraint over S. For a finite m x n array T 
with TO > 1 and n > 2 over S its [1 x 2]-higher block recoding or [1 x 2\-recoding 
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is an m X (n — 1) array f over E^^^ with entries given by 

= (r^j T^t.j+i) ; i = O, . . . , m - l, j = O, . . . , n - 2. 

We denote by S'l^^^l the set of all [1 x 2]-recodings of arrays in S and refer to 
it as the [1 x 2]-higher block recoding of 5'. The [1 x 2]-higher block recoding of 
a constraint is a constraint. This is stated in the following proposition. 

Proposition 4. Let S be a 2- dimensional constraint over S. Then S^^^^l is a 
2- dimensional constraint over Y,^^'^ . 

Remark . We may, of course, define, in a similar manner, the [s x t]-higher block 
recoding of S, for any positive integers s and t, and the [s x i]-higher block 
recoding of a 2-dimensional constraint S* is a 2-dimensional constraint. 

Proof. Set S" = S'^^^l and let (CJy,(J^) be a presentation of S with tjy = 
{V\;,E,C) and Q-h = {Vn,E,C). We construct labeled graphs Q[; = 
{VvxVv,E\C') and ^ {E,E\C') as follows. The set of edges E' is de- 
fined as 

E' = {(eo ei)ei;i''2 . g^^g^ jg ^ p^^j^ -^^ ^ 

and the labeling function £' : £" ^ E^^^ is given by 

C {eo ex) ^ {C{eo) C{ei)) ; (eo ei)e£;'. 

For every edge (eo ei)ei?' we define 

o-a;,(eoei) = (c^Sv (eo), CTev (ei)) 
Te(,(eoei) = (Tav(eo),Tav(ei)) 
o'e;^(eoei) = eo 
^^^^(eoei) = ei 

It's easy to verify that (t/y,t/^) is a presentation of S". H 

Clearly, recoding is an injective mapping, thus jS'mxnl — l'5'mx(i-i)l ^'^'^ 
positive integers m > l,n > 2. It follows that cap(S') = cap(S'["'^'^^l). The 
next proposition shows that the [1 x 2]-higher block recoding of a constraint 
with symmetric horizontal vertex-constrained strips has symmetric horizontal 
edge-constrained strips. 

Proposition 5. Let S he a 2-dimensional constraint with horizontal vertex- 
constrained strips. 

L S'l^^^l has horizontal edge- constrained strips. Moreover, S*!^^^! has sym- 
metric horizontal edge-constrained strips iff S has symmetric horizontal 
vertex-constrained strips. 

2. ^[1X2] = Vi(5[l><2])^^^(^[lx2])^ 

Proof. (P). Let m be a positive integer, G^"* ~ {vS^\Em) be the graph 
defining the vertex-constraint Hm{S) and set A = E^^^, where E is the alphabet 
of S. We define a labehng C : E„-, A™^i of the edges of cin^. For eeE„„ 
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we regard cr(e) and T(e) as mxl arrays over S, and define C(e) to be the array 
in A™^^ with entries given by 

'C(e)(i^o) = (CT(e)^,o T(e)i,o) ; i&[m]. 

It's easily verified that the word generated by every path in the labeled graph 
{G^rn\c) is the [lx2]-higher block recoding of the array formed by concatenat- 
ing the vertices along the path horizontally in sequence. It follows that iG^'^ , C) 
is a presentation of 'Hm{S^'^^'^^) . Since the labels of the edges in (G'*^^)m,'C) are 
distinct, we may identify each edge with its label, and it follows that 7im(5[^^^l) 
is an edge-constraint. Since the same graph defines both 7i„i(5) and 7im('S''^^^') 
(the former as a vertex-constraint and the latter as an edge-constraint), it fol- 
lows that HmiS) is symmetric iff HmiS^^^^^) is. This completes the proof, 
p. Clearly, S'l^^^] ^ Vi(5'ti^2])^^^(5[ix2])^ f^j. ^j^g reverse inclusion, 

let f e Vi(5'[^^^l)i8)Hi(5''^^^l) be an m X n array over S^^^. Since every row 
of r is in 7Yi(S'[^^^l), every row has a unique 1 x (n + 1) pre-image under the 
recoding map. Let F be the m x (n -I- 1) array over S, whose ith row is the 
pre-image under the recoding map of the ith row of F, for i = 0, . . . ,to — 1. 
Clearly, F is the [1 x 2]-higher block recoding of F. Thus, it suffices to show that 
T G S. For i — 0,1, . . . ,n — 1 clearly, the m x 2 array T^^i-i+i over S recodes to 
the column F^^^. By our assumption this column is in Vi{S^^^^^). Since recoding 
is injective, T^^i-i^i must be in S. Since this holds for all i = 0, 1, . . . , n — 1 and 
since TimiS) has memory 1, it follows that F G 5 and therefore F e S*!^^^!. ■ 

We can now use the method described in Section [3] to get lower bounds on 
2-dimensional constraints with symmetric horizontal vertex-constrained strips. 
This is stated in the following theorem. 

Theorem 2. Let S be a 2-dimensional constraint over an alphabet S with sym- 
metric horizontal vertex- constrained strips. Let /i>0, and a,p,q>0 be inte- 
gers, Gg = (V£,E£) be the graph defining the vertex-constraint T-li(S) (hence 
VgCTi), and cj) : (V^)'^^" — > [0, oo) be a nonnegative Junction. For an inte- 
ger n>2, let Qn be a labeled graph obtained from a deterministic presentation 
for Vn{S) by replacing each edge-label with its [lx2]-higher block recoding. Set 
An,4i = A{2{^, a, n—1, Qn, Gs), W^), where I, W^, and A{J, W^) are as defined 
in Section\^ Then 

cap(5) > log-^(^P+2g+i,0) " log^(^2g+l,0) 
~ pa 

Proof Let S' = S^^^'^l By Proposition [3 S' ^ Vi(S")(8)Hi(S"), and S' has hor- 
izontal symmetric edge-constrained strips. Since Gg has no parallel edges, we 
may identifiy each edge eGiSg with the pair (a{e),T{e)); then, with this identi- 
fication, Tii{S') = X{Gs). Also, note that g2q+p+i and ^29-1-1 are deterministic 
presentations for V2q+p{S') and V2q{S'), respectively. The result follows from 
Theorem [T] applied to 5'. ■ 
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5 Capacity bounds for axial products of con- 
straints. 



In this section we show how the method described in Section [3] can be apphed 
to axial products of certain 1-dimensional constraints. Let S and T be two 
1-dimensional constraints over an alphabet E. We wish to lower bound the 
capacity of the 2-diinensional constraint T^S. To this end, we pick a lossless 
presentation Qs — {Gs, ^s), with Gs — {Vs, Es), for S. We extend the function 
£5 to multidimensional arrays over Eg in the manner described in Section [21 
and for a set A C E*, we denote by Cg^{A)(-Eg the inverse image of A under 
this map, namely 

Cs\A) = {w e E*s : Csiw)&A} . 

The following proposition shows that we can reduce the problem of calculating 
the capacity of T^S" to that of calculating the capacity of £g^{T)(^X{Gs)- 

Proposition 6. Let S,T be two \- dimensional constraints and let X(Gs) and 
Cg^{T) be as defined above. Then 

1. Cg^{T) is a 1-dimensional constraint. 

2. cap(T(g)S') = cap(£<:^(r)(g)X(Gs)). 

Proof. [TJ Let Qt — {Vt,Et,Ct) be a presentation of T. We shall construct 
a presentation !F = {Vr, Ejr^ Cjr) of Cg^{T). The set of edges is given by 
Ey^ = {{eT,es)&ETxEs : £T{eT)=^s{es)}, and for an edge {eT,es)eEjr, 
crj=-{eT,es) = (Jg.^{eT), Ty^{eT,es) = rg^^ier) and Cjr{eT,es) = es- It is easily 
verified that Cg^{T) is presented by JF, and therefore it is a 1-dimensional 
constraint. 

[3 We set R = T®S, U = Cg\T)®X{Gs). For an array Aei?„xn, define 
Pa = {TeJ/mxri : ^s(r) — A}, we claim that 

1<\Pa\ < (11) 

Indeed, it's easily verified that an array Fei?™^" is in Pa iff for all ie[m] the 
row (Fij)"~g^ is a path in Qs that generates {Aij)"~^ . Since Qs is a lossless 
presentation of S, for every ie[m], there is at least one path in Qs generating 
{Ai_j)"~Q and at most \Vs\'^ such paths; the claim follows. Now, clearly for any 
Fet/mxn the array ^^(F) is in Rmxn- It follows that the sets Pa, for AgPmxn 
form a partition of Umxm and we have 

\Umxn\ = I^'^l- 

Therefore, by ifTTj) . we get 

|Pmxn| < |C^mxri| < |Pinxri| 

and it follows from llj that cap(P) = cap{U). ■ 

Therefore if Cs^ {T)i^X{G s) has symmetric horizontal edge-constrained 
strips, we can apply the method of Section [3] to obtain lower bounds on 
cap(T(g)S'). In this case, it also follows from Remark [T] of Theorem [1] that the 
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method of Ij for obtaining upper bounds on the capacity of the hard-square 
constraint, can be used to obtain upper bounds on cap(T(g)iS'). Proposition [3] 
present a sufficient condition for Cg^ (T)i^X{G s) to have symmetric horizon- 
tal edge-constrained strips. Here we give another stronger sufficient condition 
involving only the presentation Gs- We say that a labeled graph {G,C), with 
G = {V,E), is symmetric as a labeled graph, if there exists an edge-reversing 
matching R G TZ{G) which preserves £, that is C{R{e)) = C{e) for all e £ E. We 
assume now that Qs is symmetric as a labeled graph, and that R € Tl{Vs, Es) is 
an edge-reversing matching which preserves Cs- Since for any positive integer m 
and ei. . .em^E^ , the label £(ei). . .C{em) = C{R{ei)) . . .C{R{em)), it follows 
that ei. . .em'^C~s^{T) iff -R(ei). . .R{em)^C-^^{T). Consequently, the hypothesis 
of Proposition [3] holds and we have the following corollary. 

Corollary 1. If Qs is symmetric as a labeled graph then C^^ {T)(^X{G s) has 
symmetric horizontal edge- constrained strips. 

Since the presentation in Figure [Ta| is symmetric as a labeled graph, we 
can apply the method of Section [3] to get lower bounds on the capacity of all 
constraints T(X)EVEN for any 1-dimcnsional constraint T. 

Let S = CHG(5i) and let T = CHG(52) for integers 61,62 > 2. Let Gs = 
{Gs,Cs), with Gs — {Vs,Es) be the presentation given in Figure [Tel for 6 — 
61. Evidently, Gs is symmetric with exactly one edge-reversing matching, R : 
Es—^Es- Fix a positive integer m and let e = 6162. . .em&E'^ . Obviously, T is 
closed under negation of words (i.e., negating each symbol), and we have 

6162 . . .Cm 

^s{ei)Cs{e2)- ■ ■Cs{em)&T 
i-Cs{ei))i-Csie2)). . .(-/:5(em))eT 
CsiRiei))CsiRie2)). . .Cs{R{em))(^T 
R{ei)R{e2). . .Rie„^)eCs\T). 

Consequently, it follows by Proposition [3] that Cg^{T)(^X{Gs) has symmetric 
horizontal edge-constrained strips and we can apply the method of Section [3] to 
obtain lower bounds on the capacity of CHG(62) ® CHG(6i). 

The reader will note a similarity in the constructions in proofs of Proposi- 
tions [1] and [6] Indeed, as an alternative approach, one may be able to use the 
construction in Proposition [1] to obtain bounds on cap(S' (8) T): namely, if Gi 
and G2 are the underlying graphs of a capacity-preserving presentation (C/i, G2) 
oi S and X(Gi) (g) X(G2) has symmetric horizontal edge-constrained strips. 
However, the approach given by Proposition [5] seems to be more direct and 
simpler than the alternative approach. 

6 Heuristics for choosing 0. 

In this section, we use the notation defined in Section [31 and assume that S 
T(g)£ is a 2-dimensional constraint with symmetric horizontal edge-constrained 
strips, where £ is an edge constraint. We describe heuristics for choosing the 
function (j) to obtain "good" lower bounds on the capacity of S. 
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6.1 Using max-entropic probabilites. 

Recall that a vertex of a directed graph is isolated if no edges in the graph 
are connected to it. Note, that since Gm is symmetric, every vertex is either 
isolated or has both incoming and outgoing edges. We assume here that for 
every positive integer to, ignoring isolated vertices, Gm is a primitive graph. 
In this case, the Perron eigenvector of is unique up to multiplication by 
a scalar. Let be the right Perron eigenvector of Hm normalized to be a 
unit vector in the L2-norm. Observe, that substituting r,„ for satisfies (0]) 
with an equality. This motivates us to choose cj) so that the resulting vector 
approximates Vm- Since Gm (without its isolated vertices) is irreducible, there 
is a unique stationary probability measure having maximum entropy on arrays 
of Ti-m, namely the max-entropic probability measure on Tim- We denote it here 
by Pr*'". It is given by 

p *,m/pN ^ ('•'»)rT(r) (''"')r(r) 

for F G Smxe, for some positive integer £, and where cr(r),T(r) € are given 

by 

(cr(r))- = cr(r^_o) --019 _1 

(T(r)), = T(r,,,_i) ' ^-^J'^'A---"^ 

Let V(™) be a random variable taking values in with distribution given by 

Pr(V('")=v)=Pr*^'"{re^^xi:a(r)=v} ; v G F^". 
It's easily verified that 

Pr(V(")=v) = ((r„)v)'. (12) 

Thus approximating Pr(V*^™^ = v) and taking a square root will give us an 
approximation for {rm)v Roughly speaking, PrCV^™^ ~ v) is the probability 
of seeing the column of vertices v in the "middle" of an to x € array chosen 
uniformaly at random from Smxi, for large I. Fix integers /i > 0,Q! > 1 as 
in Section [3l and assume now that to — ruk = fj. + ka, for a positive integer 
k. For an integer 0<s<to and vectors uGV^ ^w^V^ , with lengths satisfying 
i<m—s, r<s, denote by pi™\u) and pi™\u\w) the probabilities given by 

p<rHu) = Pr(vi™|,_, = u) 
pW(u|w) = Pr(vi™|,_,=u|vi™,L_,=w). 

Then by the chain rule for conditional probability we have, for any vector vg Vg", 

fc-i 

Pr(V(")=v) =p[,"^(vo:^_l) Y[p^^\c,i^^,+^a■.^,+ ^^+l}a-lWo■.^.+^a-l)■ 

i=0 

A plausible way to approximate Pr(V = v), is by treating V as the outcome 
of a Markov process. Here we use a Markov process with memory /i, and 
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assume that ps(u|w) can be "well" approximated by ps(u|wr_p:r-i), for vectors 
ugV^, wgV^, with r,£ as above, and r>/i. Using this approximation we get 

fc-i 

We hypothesize that for fixed vectors u£V^ ,w^V^ , as m gets large, the 

conditional probabilities pi™'' (u|w), for 0^s<Cto— 1, are "approximately equal" 
to the value when s is in the "middle" of the interval [0, m—1]. We hypothesize 
that this holds for "most" of the integers s in that inteval and moreover that this 
middle value converges as m gets large. Accordingly, we try to approximate the 
conditional probability pi™\u\w) by the conditional probability found in the 
"middle" of a "tall" horizontal strip. More precisely, we fix an integer S > 0, set 
uj=2S+^j,+a, and approximate pi™^(u|w) by pg^^{u\Mv). We also approximate 
Pq'"-'(w) by Pq'^''(w). This gives us 

fe-i 

Pr(V('")=v) Wp[,"^(vo:p_l) [|4+V^A'+'a:A'+(«+l)"-l|Vi:a:p+»a-l), 

i=0 

which, by (fT2| . implies that 



(l"m)v ~ V^'o"^('^0^*'-l) n V^^+U^A'+^a:A^+(^+l)a-l|v^a:M+^a-l) ; VGF™". 
i=0 

Set i^m^lVfl™, and denote by r„jj.eM^"'fc the nonnegative real vector with en- 
tries indexed by y"*" and given by the RHS of equation dTS]). Let : 
[0, oo) be given by 

0(u) = v'p5^^(um:.+«-i|uo:p-i) ; ue(y£)''+", (14) 

and let z^^ e ]R^™'= be the vector with entries indexed by V^"*" and given by 
Setting Xmj, = z^^ , we obtain 

Now for mk>u), if veV^'*" is not an isolated vertex in Gm^, then clearly, Vq-.u-i 
is not an isolated vertex in Gu: as well. Therefore (rt^)vo^-i >0, which implies 
thatp^'"^(vo:c:j_i)>0 and thus p[,"^(vo:^-i)>0. Letpmin pi'^n = min{p^'^^ (w) : 
weV^ and (vir)>0}. It follows that for aU vertices vG (Fg)"" of G™^ that 
are not isolated, we have 

Now, for any positive integer £ and i^„i^xl-real vector, the product y'^H^^y 
depends only on the values of the entries of y indexed by non-isolated vertices 
of Grrik ■ Consequently, we may write 

n ■ X* /T^ X < r * f < x* /T^ x 
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for all positive integers I. Taking the log, dividing by to/j, and taking the limit 
as fc approaches infinity, we obtain 

fc^oo mfc fc— too mfc ' 

where by Lemma [1] the limit in the RHS exists. Thus, choosing ^ as given 
by (|14p and computing the lower bound by the method described in Section [3] is 
equivalent to computing the limit of the lower bound in (U, with iVi substituted 
for Xm, as m approaches infinity. If approximates well enough, we expect 
to get good bounds. Note that we may use the heuristic described here even 
for constraints for which the graphs are not always irreducible. In this 

case, the geometric multiplicity of the Perron eigenvalue may be larger than 1, 
and there may be more than one choice of the vector in the computation of 
Regardless of our choice, we will get a nonncgative function and a 
lower bound on the capacity. In Section [7] we show numerical results obtained 
using the heuristic described here for several constraints. 



6.2 General optimization. 

We may also use general optimization techniques to find functions which 
maximize the lower bound on the capacity. Fix integers /i>0 and p, (7,ck>0, 
and for a positive integer ^, set Vi = [VsY . In this subsection, we identify a 
function (/):P^,+q^M with a real vector (/()€Ml-^f+°l indexed by V^j^a] for each 
jeI?;^+Q we identify <j){]) with the entry (j)y For a positive integer n, let Qn 
be a deterministic presentation for Vn(.S), X„ — a, n, t/„, G^), and for a 
function (j) : I?^i+„^[0, oo), set An^ff, = Ailn.yV^,). Observe that for a scalar 
c e [0, oo), A„^c</. = c^^n,0. It follows that using c<j) in place of </> in equation ([9|) 
of Thereom [1] does not change the lower bound. Consequently, (as (j) cannot 
be the constant function), it's enough to consider functions whose images 
(of all vectors in (Vf:)''^") sum to 1. We thus have the following optimization 
problem. 

maximize (log A(A2g+p,0) - \ogX{A2q,4,)) / {pa) 
subject to > 0, (15) 
0-1 = 1, 

where and 1 denote the real vectors of size |I?^+q| with every entry equal to 
and 1 respectively, and for two real vectors of the same size, t, r we write t > r 
or t>r if the corresponding inequality holds entrywise. 

Finding a global solution for a general optimization problem can be hard. 
We proceed to show that if we replace the constraint (/)>0 with (/»0 in 
thereby changing the feasable set and possibly decreasing the optimal solution, 
it can be formulated as an instance of a particular class of optimization problems 
known as "DC optimization" which may be easier to solve. Let d be a positive 
integer. A real- valued function / : R*^ ^ M is called a, DC (difference of convex) 
function, if it can be written as the difference of two real- valued convex functions 
on W^. An optimization problem of the form 

maximize f{x) 
subject to x&X, 

h,{x) < ; i = 0, 1,. . . 
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where X C M'* is a convex closed subset of and the functions f,ho,...,h^ 
are DC functions, is called a DC optimization or DC programming problem. See 
[5] and the references within for an overview of the theory of DC optimization. 

A nonnegative function / : R'' — )■ [0, oo) is called log-convex or superconvex, 
if either /(t)>0 for all t£M.'^ and log / is convex in R'', or /=0. A log-convex 
function is convex, and in ^12i| , it is shown that the class of log-convex functions 
is closed under addition, multiplication, raising to positive real powers taking 
limits, and additionally that for a square matrix A(t) = (aij(t)) whose entries 
are log-convex functions Oij : R'' [0, oo), the function t — > X{A{t)) is log- 
convex as well. 

Now, observe, that for a positive integer n, every entery of An^^ is a quadratic 
form in the entries jsP^+a, with nonnegative integer coefficients. Such 
a function is generally not log-convex. To fix this, we perform the change of 
variables (j) — , where is a real- valued function : V^+a R- Note 
that by doing so, we added the constraint 0>O. Since every entry of cj) is now 
positive, we may replace the constraint (j) ■ \ — 1 hy the constraint 0(vo) = 1 
or equivalently ■(/'(vq) — 0, for some fixed VQ^V^+a- Problem with the 
additional constraint 0>O, can now be rewritten as 

maximize (log \{A2q+p^e^ ) - log A(A2,,e* )) / ijpa) 
subject to V'(vo) =0. 

Obviously, we may substitue the maximization problem constraint, V'(vo) — 0, 
above into the objective function, thereby reducing the number of variables by 
1; however, this is not relevant for the discussion, so, for simplicity, we do not 
do so here. Now, for a positive integer n, the entries of the matrix A„ are of 
the form 

g'/'(wfc,i,j)+'/'(ufc,i,j)^ 

k=l 

where qij are nonnegative integers, and '^k,i,j and vLk,i,] are vectors in P^+a, 
for all i,j&Vx^ and integers l<k<qij. It can be verified that a function of 
such a form is log-convex in ip. It follows that the function '0^A(A„_eV") for 
ip g Rl^t'+ol is log-convex as well. Therefore either A(A„ e^)=0, or X{An^e^)>0 
for all ijj G RI^''+°I. In particular, for ip=0 the matrix A„_i is the adjacency 
matrix of the graph I„. Since X„ is deterministic, and for every nonnegative 
integer £, the set of labels of its paths of length £ is Sgaxn, it follows that 
(1/a) log A(A,i^i)=cap(V„(5')) > cap(S') (the latter inequality follows from 
Hence, if cap(S')>— oo (or equivalently cap(5')>0), then A(A„ g^)>0 for all tp £ 
R'^f+°l and logA(A„ g^) is a convex function of ip- Clearly cap(S')>— oo iff. 
SmxriT^^, for all positive integers m,n. We thus obtain the following theorem. 

Theorem 3. Let S be a constraint such that for all positive integers m, n, 
SmxriT^^ then Problem ([T5|) is a DC optimization problem. 

7 Numerical results for selected constraints. 

In this section we give numerical lower bounds on the capacity of some 2- 
dimensional constraints obtained using the method presented in the sections 
above. The constraints considered are NAK, RWIM, EVEN®^, and CHG(3)®2^ 
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Table [T] summarizes the best lower bounds obtained using our method. For 
comparison, we provide the best lower bounds that we could obtain using other 
methods. We also give upper bounds on the capacity of these constraints ob- 
tained using the method of [T] . Table [H shows the lower bounds obtained using 
our max-entropic probability heuristic for choosing (jj, described in Section [6. II 
Table [3] shows the lower bounds obtained with our method by trying to solve 
the optimization problem described in Section [5?^ In this, we did not make use 
of the DC property of the optimization problem; instead, we used a generic sub- 
optimal optimization algorithm whose results are not guarenteed to be global 
solutions. Utilizing special algorithms for solving DC optimization problems 
may give better lower bounds. The rightmost column of each of these tables 
shows the lower bound calculated for the same values of p and q using the 
method of [T]. The largest lower-bound obtained for each constraint is marked 
with a '*'. In the next subsections we give remarks specific to some of these 
constraints. 

The numerical results were computed using the eigenvalue routines in Matlab 
and rounded (down for lower bounds and up for upper-bounds) to 10 decimal 
places. Given accuracy problems with possibly defective matrices, we verified 
the results using the technique described in [TH Section IV] . 

7.1 The constraint RWIM 

Observe that this constraint has both symmetric horizontal and symmetric 
vertical vertex-constrained strips. Thus, we can apply our method in the 
vertical as well as the horizontal direction to get lower bounds. Clearly, 
cap(RWIM*) = cap(RWIM), so we can obtain additional lower bounds on 
cap(RWIM) by using our method to get lower bounds on cap(RWIM*). Some 
of these bounds are given in Tables [U and [31 

7.2 The constraint EVEN®^ 

We used the reduction described in Section [5] with ^/even being the presentation 
of EVEN given in Figure fTal to get lower bounds on the capacity of EVEN®^. 
Table |3] gives the results obtained with our method using the optimization de- 
scribed in Section We also used the method with the max-entropic proba- 
bility heuristic of Section 16.11 and the results are given in Table [H 

7.3 The constraint CHG(fe)^2 

For this constraint, the case b=l is degenerate. Indeed, there are exactly two 
mxn arrays in CIIG(l)®^ for all positive integers m and n, and consequently, 
cap(CHG(l)'^2)^0. For 6=2, we show in Theorem H in Section [5] that the 
capacity is exactly 1/4. For 6=3, we used the reduction of SectionOwith ^chg(3) 
being the presentation of CIIG(3) given in Figure [Tel to get lower bounds on 
the capacity of CIIG(3)®^. Table [31 gives the results obtained with our method 
using the optimization described in Section [621 We also used the method with 
the max-entropic probability heuristic of Section 16.11 and the results are given 
in Table [3 
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Table 1: Best known lower bounds on capacities of certain constraints. 



Constraint 


Prcv. best lower bound 


New lower bound 


Upper bound 


NAK 
RWIM 

EVEN®^ 
CHG(3)®2 


0.4250636891* 
0.5350150t 
0.4385027973* 
0.4210209862* 


0.4250767745 
0.5350151497 
0.4402086447 
0.4222689819 


0.4250767997* 
0.5350428519* 
0.4452873312* 
0.5328488954* 



*Calculated using the method of [T]. 
Appears in [T7] . 



8 Exact Computation 

While it seems difficult to compute the capacity exactly for constraints such 
as EVEN®® and CHG(3)®®, we can compute the capacities of constraints in 
related families: 

Theorem 4. For all positive integers J), 

cap(CHG(2)«®) = ^. 
Theorem 5. For all positive integers D , 

cap (odd®®) 

Proof of Theorem\^ Let S = CHG(2)®®. We first show that cap(5)>l/2®. 
Let r(°\r(^) be the ©-dimensional arrays of size 2x2x. . .x2 with entries in- 
dexed by {0,1}® and given by 

(rW)_ = (-1)'+Ji ; jG{0,l}®, 

where as usual 1 denotes the ©-dimensional vector with every entry equal to 
1. Observe that the sum of every row of both of these arrays is zero. Now, let 
n be a positive integer. For any ©-dimensional array of size nxnx. . .xn with 
entries in {0, 1}, it can be easily verified that replacing every entry equal to 
with r(°) and every entry equal to 1 with P'^' results in a ©-dimensional array 
of size 2nx2nx. . .x2n that satisfies S. It follows that |S'2nx2nx...x2n |>2" for 
all positive integers n, which implies cap(S')>l/2® . 

We now show that cap(S')<l/2® . For a positive integer n>2, denote by 
A/"i°^ the set of all even integers in {0, 1, . . ., n— 2} and by A/'i^'' the set of all odd 
integers in {0, 1, . . ., n— 2}. We shall make use of the following lemma. 

Lemma 2. Fix a positive integer n>2, and let (aj)"^Q^ C {-|-1, —1} be a sequence 
of length n. Then oq. . .a„_ieCHG(2) if and only if at least one of the following 
statements hold. 

1. For all i^Mn\ CLi=—ai+i- 

2. For all iGj^ji'\ ai——ai+i. 
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Proof. We first show the "if" direction. Let {ai)"^Q C {+1,-1} be a sequence 
for which at least one of statements lll2l of the lemma holds. Then clearly, for any 
integers 0<i<j<n, all the terms in the sum '^k, with the possible exception 

of the first and last terms, cancel. Therefore \ J2k=i'^k\ < + \aj\ — 2 and 
ao...a„_iGCHG(2). 

As for the "only if" direction, let (ai)"jQ^C {+1, —1} such that ap. . .a„_i G 
CHG(2), and consider the presentation of the CHG constraint given in Figure [Tel 
for 6 = 2 (and vertices {0,1,2}). Let {ei)"~Q be a path in this presentation 
generating oq. . .a„_i. It's easily verified that if a{ei) — 1, for some z£[n— 1], 
then aj = — flj+i for all integers i<j<n—2 such that j=i (mod 2). Evidently, 
either cr(eo) = 1 and so statement [T] holds, or cr(ei) = 1 impliying statement [21 

■ 

We now return to the claim that cap(S')<l/2® . Fix a positive integer n>2. 
For an integer l<i<2), let e(*'e{0, 1}®, be the vector, indexed by {1, 2, . . .,2)}, 
containing 1 in its zth entry and everywhere else and let j7iC[n]® denote the 
subset of all the vectors indexed by {1,2, ...,£>} with a in the ith entry. For a 

vector jeJi, the sequence (j + /ce(*))^_Q, is a sequence of indices of entries of a 
row in direction z of a J)-dimensional nxnx. . .xn array, and we shall say that 
it is a sequence in direction i. Let R(n, S) be the set of all such sequences for 
all integers l<i<!D and vectors jSjTi, and let re{0, 1}I^("'^)I be a binary vector 
indexed by R(n,S). For the purpose of this proof, let us refer to a sequence 
{ai)^~Q C {+1, —1} as a phase-0 sequence if statement [1] of Lemma [21 holds, and 
as a phase- 1 sequence if statement [H holds (note that a sequence may be both 
a phase and a phase 1 sequence). Also, we denote by ^(r)C{ + l, —1}* , the 
set of all S-dimensional arrays F of size nxnx. . .xn for which the row Fg is a 

phase-rg sequence, for all ^ = ( j + ke'-'^^)^_^ GR(ri,X)). Then by Lemma[2 we 
have 

^„x...x„ = lJ-^W- (17) 

r 

We shall give an upper bound on the size of -4(r). For a vector ve[ri]®, denote 
by p{i, v) the unique sequence in direction i in R(n, D) that has v as one of its 
elements. Let T^^i : [n]®-^Z® be given by: 

rr f \ / v + e('' if EE r„(i.v) (mod 2) i© / n 

' I ^ 6 otherwise 

Next, we define the undirected graph Gr = {V,Er) (without parallel edges), 
with vertices given by 

V^[nf, 

and edges given by 

Er = {u — V : u,veV and v = Tr^i{u.) for some integer l<i<D} , 

where u — v denotes the undirected edge connecting vertices u, v. It's easy to 
verify that an array re{ + l, —1}* of size nxnx. . .xn is in A{r) iff for every 
edge u — vEEr, it holds that Fu = — Fv Figure [31 shows an example of Gr for 
S) = 2. 

Let Ci,...,Ce be the connected components of Gr, and let v*^^) , . . ., v^^' 
be arbitrary vertices such that v^'^eGi, for i — 1,2,. . .,£. It follows that for 
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Figure 3: Example of the graph Gr for J) = 2, n = 6. Each entry of r corre- 
sponding to a row (column) is written to the right of it (below it). The index 
of each row (column) is written to its left (above it). 



every vector b = (6i, . . ., 6^)e{+l, — 1}^, there exists at most one array rG^(r) 
satisfying T^(i) = bi for all 2, . . .,£}, and consequently, |^(r)|<2^ (in fact, 
while this is not needed for the proof, such an array r€:^(r) does exist, for any 
choice of b, since each Cj is bipartite; thus |^(r)| = 2^). 

Now, let u = («!,..., Ms))G([n] \ {0, n— 1})® be a vertex in the "interior" of 
Gr- We show that the connected component of Gr containing u has at least 
2® vertices. To this end, we match each word w = wiW2- ■ ■Wj)G{Q, 1}®, with a 

sequence of vertices (''■'^'■'^)^q ^V", defined recursively by 




^(w,j-i) if j > and = . 

Tr,j (tt^'"''^- 1) ) if j > and wj = 1 



It's easy to verify that since every l<Ui<n—2, the sequence is well-defined and 
indeed (7r(^'^^)|Lo C [n]^ . Clearly, the sequence is contained entirely in the 
connected component containing u, and so this component contains the vertex 
^(w,»)^ Write 7r(^'®)=(4"''®\...,4"''®^). Then for z=l, 2, . . ., 2), it holds 
that 7r-^'^^=Ui if Wi=0, and 7r-^'^^=Ui±l if Wi=l. Therefore, for two distinct 
words w. w'g{0,1}®, the vertices tt'^'^'®^ and tt'^'^ arc distinct as well, and 
consequently there are 2® such vertices. Thus, the connected component of Gr 
containing u has at least 2® vertices. 

It follows that there arc at most n®/2® connected components of Gr con- 
taining a vertex in {1, 2, . . ., n— 2}® . There are at most — (n— 2)® connected 
components not containing a vertex in {1, 2, . . ., n— 2}® and hence the total 
number of connected components, i, in Gr satisfies i<n^ + — (n— 2)®. 
Hence 

|^(r)|<2""/2"+""-("-2)". 
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Since there are 2®"^ binary vectors rG{0, we obtain from (fT7|l 

I < ^|^(r)| 

r 

^ 2"^(l/2®+l-(l-2/n)»)+2n^-i 



and the result foUows from H]). ■ 

Proof of Theorem\^ Let S be the S-dimensional constraint ODD®®. We first 
show cap(S')>l/2. For an integer n, let Bn ^ [2n]® be the set of all vectors in 
[2n]® whose entries sum to an even number, and let An be the set of all binary 
S-dimensional arrays T of size 2nx2nx. . .x2n, with entries satisfying (r)j = 
for all i^zBn- Then the number of zeros between consecutive 'I's, in any row 
of an array in An is odd since it must be of the form i—j^l for some integers 
i,j-either both odd, or both even. Thus, all such arrays satisfy the constraint 
S, and since = 2(2")"-l^.l = 2(2")"/2, we have |52„x2„x...x2„| > 2(2")"/2 
for all positive integers n, which implies cap(S')>l/2. 

It remains to show that cap(S')<l/2. For a positive integer d, let T^"*^ = 
ODD®'*. Since for any d positive integers toi, . . ., md, 



(rp{d)\ _ (rpid+l)\ 

)miX...xmd — ) 



)mi x...xmd x 1 



it follows from ^ that cap(ODD®'*) is non-increasing in d. Thus, it's enough to 
show cap(S')<l/2 for D = 1. Let n be a positive integer. It can be easily verified 
that any 1-dimensional array rsODD„ with entries indexed by [n], satisfies 
either Tj = for all even integers or Tj — for all odd integers jG[n]. It 

follows that |ODD„|<2r"/2T+2L"/2J which imphes the desired inequality ■ 
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Table 2: Lower bounds using max-entropic probabilty heuristic (Section 16. ip . 



Constraint 


6 




a 


P 


9 


Lower bound 


Using [5 


NAK 


3 


1 


1 


1 


5 


0.4250766244 


0.4248771038 




3 


1 


1 


2 


4 


0.4250766446 


0.4249055702 




6 


1 


1 


1 


5 


0.4250767227 


0.4248771038 




3 


3 


4 


1 


5 


0.4250767590 


0.4248771038 




7 


1 


1 


1 


5 


0.4250767617 


0.4248771038 




3 


1 


1 


2 


6 


0.4250767647 


0.4250636891 




3 


1 


4 


1 


5 


0.4250767733 


0.4248771038 




5 


1 


1 


1 


5 


0.4250767744 


0.4248771038 




3 


1 


4 


2 


6 


0.4250767745 


0.4250636891 




3 


1 


2 


2 


6 


0.4250767745 


0.4250636891 


RWIM 


3 


1 


3 


1 


6 


0.5350147968 


0.5235145644 




1 


1 


1 


3 


6 


0.5350148753 


0.5318753627 




3 


2 


2 


1 


5 


0.5350148814 


0.5160533001 




3 


1 


2 


2 


6 


0.5350149069 


0.5337927416 




2 


1 


2 


2 


6 


0.5350149071 


0.5337927416 







1 


2 


2 


6 


0.5350149136 


0.5337927416 




2 


2 


2 


1 


5 


0.5350149271 


0.5160533001 




1 


1 


2 


2 


6 


0.5350149462 


0.5337927416 




1 


1 


3 


1 


6 


0.5350149525 


0.5235145644 




1 


1 


1 


1 


7 


0.5350149707 


0.5280406048 


RWIM'^ 


4 


1 


3 


2 


4 


0.5350145937 


0.5350144722 




1 


1 


1 


1 


5 


0.5350146612 


0.5350149478 




4 


2 


1 


1 


4 


0.5350147212 


0.5350142142 




3 


1 


1 


1 


5 


0.5350147328 


0.5350149478 




5 


1 


1 


1 


5 


0.5350147619 


0.5350149478 




2 


2 


1 


1 


4 


0.5350147969 


0.5350142142 




4 


1 


1 


1 


5 


0.5350148255 


0.5350149478 




2 


1 


1 


1 


5 


0.5350148449 


0.5350149478 







1 


1 


1 


5 


0.5350148814 


0.5350149478 







2 


1 


1 


4 


0.5350148980 


0.5350142142 


EVEN^^ 


3 


2 


1 


1 


3 


0.4383238232 


0.4347423815 




3 


1 


1 


1 


4 


0.4383243738 


0.4367818624 




3 


1 


3 


2 


3 


0.4383632350 


0.4356897662 




3 


1 


2 


4 


3 


0.4383838005 


0.4364303826 




3 


1 


1 


2 


4 


0.4384647082 


0.4371709990 




3 


1 


3 


3 


3 


0.4384906740 


0.4360537982 




3 


1 


2 


1 


4 


0.4385448358 


0.4367818624 




3 


1 


2 


2 


4 


0.4386655840 


0.4371709990 




3 


1 


3 


1 


4 


0.4387455520 


0.4367818624 


CHG(3)^^ 








1 


1 


2 


0.4188210386 


0.4101473707 










1 


1 


4 


0.4222689819* 


0.4197053158 



*Best lower bound. 
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Table 3: Lower bounds using optimization (Section 16. 2p . 



Constraint 




a 


P 


9 


Lower bound 


Using 1 


NAK 


2 


1 


2 


4 


0.4250767692 


0.4249055702 




1 


2 


1 


5 


0.4250767736 


0.4248771038 




1 


1 


3 


4 


0.4250767737 


0.4248960814 




1 


2 


1 


3 


0.4250767739 


0.4224650194 




1 


1 


4 


4 


0.4250767739 


0.4249674993 




1 


1 


5 


4 


0.4250767740 


0.4249783192 




1 


1 


6 


4 


0.4250767741 


0.4249995626 




1 


2 


3 


3 


0.4250767743 


0.4244240822 




1 


2 


6 


3 


0.4250767744 


0.4247979797 




1 


2 


2 


5 


0.4250767745* 


0.4250294285 


RWIM 


1 


1 


1 


3 


0.5350147515 


0.4832292495 




1 


1 


2 


3 


0.5350148675 


0.5300373650 




1 


1 


3 


3 


0.5350149371 


0.5212673183 




1 


1 


1 


4 


0.5350150805 


0.5037272248 




1 


1 


2 


4 


0.5350151001 


0.5318663054 




1 


1 


3 


4 


0.5350151123 


0.5265953036 




1 


1 


1 


5 


0.5350151372 


0.5160533001 




1 


1 


2 


5 


0.5350151410 


0.5330440001 




1 


1 


2 


6 


0.5350151491 


0.5337927416 




1 


2 


1 


5 


0.5350151497* 


0.5160533001 


RWIM"^ 


1 


2 


4 


3 


0.5350151364 


0.5350130576 




1 


2 


3 


4 


0.5350151377 


0.5350146307 




1 


2 


5 


3 


0.5350151392 


0.5350134356 




2 


1 


1 


4 


0.5350151405 


0.5350142142 




1 


1 


1 


5 


0.5350151442 


0.5350149478 




1 


2 


1 


4 


0.5350151465 


0.5350142142 




1 


2 


1 


5 


0.5350151481 


0.5350149478 




1 


2 


2 


4 


0.5350151482 


0.5350144722 




1 


3 


1 


4 


0.5350151483 


0.5350142142 


EVEN*^^ 


1 


1 


1 


3 


0.4395381520 


0.4347423815 




1 


1 


2 


3 


0.4397347451 


0.4356897662 




1 


1 


1 


4 


0.4402086447* 


0.4367818624 


CHG(3)^^ 





1 


1 


2 


0.4189237100 


0.4101473707 







1 


2 


2 


0.4197037681 


0.4182017399 







1 


3 


2 


0.4201450063 


0.4176642274 







1 


1 


3 


0.4210954837 


0.4165892023 







1 


2 


3 


0.4214748454 


0.4210209862 








1 


4 




0.4197053158 



*Best lower bound. 
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